Multi-target Attachment for Surgical Instrument Tracking

The pose estimation of a surgical instrument is a common problem in the new needs of medical science. Many instrument tracking methods use markers with a known geometry that allows for solving the instrument pose as detected by a camera. However, marker occlusion happens, and it hinders correct pose estimation. In this work, we propose an adaptable multi-target attachment with ArUco markers to solve occlusion problems on tracking a medical instrument like an ultrasound probe or a scalpel. Our multi-target system allows for precise and redundant real-time pose estimation implemented in OpenCV. Encouraging results show that the multi-target device may prove useful in the clinical settin

Three-dimensional multimodal medical imaging system based on freehand ultrasound and structured light

We propose a three-dimensional (3D) multimodal medical imaging system that combines freehand ultrasound and structured light 3D reconstruction in a single coordinate system without requiring registration. To the best of our knowledge, these techniques have not been combined as a multimodal imaging technique. The system complements the internal 3D information acquired with ultrasound with the external surface measured with the structured light technique. Moreover, the ultrasound probe’s optical tracking for pose estimation was implemented based on a convolutional neural network. Experimental results show the system’s high accuracy and reproducibility, as well as its potential for preoperative and intraoperative applications. The experimental multimodal error, or the distance from two surfaces obtained with different modalities, was 0.12 m

3D Multimodal Medical Imaging System Based on Freehand Ultrasound, Structured Light, and Stereo Vision Tracking

The most used medical imaging techniques are Magnetic Resonance Imaging (MRI), Com- puted Tomography (CT), and Ultrasound (US). Typically, they provide 2D information even though the structure of a patient is in 3D. We can acquire volumetric data with US but this 3D US data does not provide information about the external topography of the patient. We can complement 3D US data with the external surface of the subject or region of interest for a better interpretation by the physicians. In this thesis, we propose a 3D multimodal medical imaging system that combines freehand ultrasound and structured light 3D reconstruction in a single coordinate system without requiring registration. With this method, we can acquire the internal structure in 3D with freehand ultrasound, and the external features with structured light. To the best of our knowledge, these techniques have not been combined before as a multimodal imaging system. We can use our proposed multimodal technique as a tool for preoperative and intraoperative tasks or even for diagnostic and treatment planning. Our proposed method shows the potential as a navigation system because we acquire the 3D pose of the ultrasound probe and the 3D surface topography in the same coordinate frames. This enables us to show the external surface with a model of a probe and with the US slices acquired in real-time and 3D. For pose estimation, we propose an approach based on deep learning, a planar marker of three circles, and a calibrated stereo vision system. We compare this approach with a method implemented using classical computer vision techniques. Finally, we carried out an evaluation study of different triangulation methods and their influence on the reconstruction error for structured light systems. We discuss what triangulation approach to use based on the conditions and required application

MarkerPose: Robust Real-time Planar Target Tracking for Accurate Stereo Pose Estimation

Despite the attention marker-less pose estimation has attracted in recent years, marker-based approaches still provide unbeatable accuracy under controlled environmental conditions. Thus, they are used in many fields such as robotics or biomedical applications but are primarily implemented through classical approaches, which require lots of heuristics and parameter tuning for reliable performance under different environments. In this work, we propose MarkerPose, a robust, real-time pose estimation system based on a planar target of three circles and a stereo vision system. MarkerPose is meant for highaccuracy pose estimation applications. Our method consists of two deep neural networks for marker point detection. A SuperPoint-like network for pixel-level accuracy keypoint localization and classification, and we introduce EllipSegNet, a lightweight ellipse segmentation network for sub-pixel-level accuracy keypoint detection. The marker’s pose is estimated through stereo triangulation. The target point detection is robust to low lighting and motion blur conditions. We compared MarkerPose with a detection method based on classical computer vision techniques using a robotic arm for validation. The results show our method provides better accuracy than the classical technique. Finally, we demonstrate the suitability of MarkerPose in a 3D freehand ultrasound system, which is an application where highly accurate pose estimation is required. Code is available in Python and C++ a

A Structure-from-Motion Pipeline for Generating Digital Elevation Models for Surface-Runoff Analysis

Digital Elevation Models (DEMs) are used to derive information from the morphology of a land. The topographic attributes obtained from the DEM data allow the construction of watershed delineation useful for predicting the behavior of systems and for studying hydrological processes. Imagery acquired from Unmanned Aerial Vehicles (UAVs) and 3D photogrammetry techniques offer cost-effective advantages over other remote sensing methods such as LIDAR or RADAR. In particular, a high spatial resolution for measuring the terrain microtopography. In this work, we propose a Structure from Motion (SfM) pipeline using UAVs for generating high-resolution, high-quality DEMs for developing a rainfall-runoff model to study flood areas. SfM is a computer vision technique that simultaneously estimates the 3D coordinates of a scene and the pose of a camera that moves around it. The result is a 3D point cloud which we process to obtain a georeference model from the GPS information of the camera and ground control points. The pipeline is based on open source software OpenSfM and OpenDroneMap. Encouraging experimental results on a test land show that the produced DEMs meet the metrological requirements for developing a surface-runoff model. © Published under licence by IOP Publishing Ltd. This work has been partly funded by Universidad Tecnológica de Bolívar project (FI2006T2001). The authors thank Direccion de Investigaciones Universidad Tecnologica de Bolivar for their support